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INTRODUCTION 


The  trend  in  fighter  cockpits  has  been  to  provide  flight  reference  information  on  Head-Up 
Displays  (HUD)  i.e.  pitch  ladder  formats.  The  HUD  is  one  place  to  present  relevant  flight  information 
where  it  can  be  seen,  while  allowmg  the  pilot  to  focus  his  attention  outside  the  cockpit.  Because  the 
HUD  is  superimposed  on  the  outside  world,  a  problem  arises  when  trying  to  determine  which  way  is 
upright.  USAF  Colonel  G.B.  McNaughton  (198S)  stated  that  no  clear  distinction  can  be  made  between 
sky  and  surface  on  the  HUD,  so  pilots  must  look  to  see  if  the  pitch  bars  are  solid  (nose  high)  or  dashed 
(nose  low).  He  further  states  that  this  method  of  conveying  aircraft  attitude  information  is  far  from 
optimal  and  occasionally  results  in  HUD  related  disorientation. 

The  inability  of  aircrews  to  recover  from  unusual  attitudes  has  become  a  rntyor  concern  in 
recent  years.  Disorientation  is  primarily  caused  by  the  display  formats  and  symbology  which  do  not 
allow  for  quick  and  clear  interpretation  by  the  pilot.  Because  they  have  1:1  pitch  scales,  USAF  pitch 
ladder  formats  provide  a  pitch  display  that  is  limited  by  the  vertical  field  of  view.  This  display  can  move 
very  rapidly  and  be  difficult  to  use  in  unusual  attitude  recoveries  (Burns,  1986). 

Despite  problems  associated  with  unusual  attitudes,  the  HUD  has  many  benefits.  According 
to  Burns  (1986),  the  primary  advantage  of  the  HUD  is  that  it  presents  information  collimated,  focused, 
and  overlayed  on  the  outside  world.  The  HUD  also  gives  pilots  the  capability  to  fly  at  night  using 
real-time  low-light-level  telewsion  or  Forward  Looking  Infrared  (FLIR)  sensor  video  integrated  with 
Terrain  Followmg  Radar  (TFR).  Other  advantages  include  an  improved  ability  to  conduct  head-up 
operations  and  navigation,  as  well  as  improved  weapon  system  effectiveness. 

At  the  direction  of  the  F-16  System  Program  Office  (SPO),  the  Crew  Station  Evaluation 
Facility  (CSEF)  evaluated  the  effects  of  certain  symbolic  codes  presented  on  the  HUD.  Specifically, 
the  effects  of  three  factors  were  examined;  (1)  pitch  bar  articulation  (sloping  pitch  bars  that  funnel  to 
the  horizon),  (2)  pitch  number  location  and  (3)  horizon  line  length.  The  SPO’s  goal  was  to  find  out 
which  set  of  HUD  symbology  enhanced  pilot  performance  the  most. 

Evaluation  Phases 

The  HUD  symbology  was  evaluated  in  an  F-16  simulator.  Evaluations  were  based  on  the 
pilot’s  ability  to  recover  from  unusual  attitudes.  Objective  (performance)  data  included  pilot  reaction 
time,  recovery  time,  correct/incorrect  stick  input  (nose  low  attitudes  only)  and  altitude  gain/loss  for 
each  configuration.  There  were  two  phases  to  the  evaluation. 

The  Phase  1  evaluation  involved  three  pitch  scale  formats  and  two  number  configurations.  All 
sue  configurations  evaluated  in  this  phase  had  a  horizon  line  that  extended  the  entire  width  of  the 
HUD.  The  purpose  of  Phase  I  was  to  determine  which  pitch  scale/number  configuration  resulted  in  the 
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most  acceptable  recoveries  from  an  unusual  attitude.  This  phase  was  concerned  with  the  effects  that 
articulated  pitch  bars  and  pitch  number  location  had  on  unusual  attitude  recoveries.  Both  objective 
and  subjective  data  were  collected  in  this  phase. 

Phase  II  examined  the  effects  of  horizon  line  length  on  unusual  attitude  recoveries.  Two 
formats  were  evaluated.  The  pitch  scale  and  number  configuration  were  the  same  as  the  Baseline  F-16 
Block  40  HUD.  The  two  horizon  lines  evaluated  were  (1)  the  F-16  Block  40  horizon  line  and  (2)  a 
horizon  line  that  extended  the  entire  width  of  the  HUD.  Objective  and  subjective  data  were  collected 
during  this  phase. 
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PHASE  1 


METHOD 


SUBJECTS 

Twelve  Tactical  Air  Command  (TAG)  F'16  pilots  took  part  in  the  evaluation.  The  pilots 
averaged  663  hours  of  flying  time  in  the  F-16  (standard  deviation  ■  789.3)  and  1,638  hours  of  total  flying 
time  (standard  deviation  «■  667.6). 

APPARATUS 

Facility.  The  study  was  conducted  at  the  CSEF  which  is  an  U.S.  Air  Force  simulation  facility 
that  belongs  to  the  Aeronautical  Systems  Division  (ASD)  of  Air  Force  Systems  Command,  at 
Wright-Patterson  AFB,  Ohio.  The  CSEF  government  personnel  are  assigned  by  the  Crew  Systems 
Division  (ASD/ENEC).  The  facility  performs  human  engineering  evaluations  in  support  of  a  variety  of 
System  Program  Offices  (SPOs). 

F46  Simulator.  The  Ft16C  simulator  was  constructed  using  a  salvaged  single-seat  F-16 
cockpit,  and  includes  the  seat  and  canopy  assemblies.  The  all  digital  design  includes  two  4X4  inch 
monochromatic  Multi-Function  Displays  (MFDs),  a  Wide  Field-of-View  raster  video  HUD,  an 
Integrated  Control  Panel,  a  Data  Entry  Display,  Hands-on  Throttle  and  Stick  controls,  and  the 
LANTIRN  avionics  suite.  The  side  control  stick,  throttle  and  flight  controls  are  actual  F-1.6 
components.  All  other  instruments  and  displays  are  simulated  using  locally  available  equipment.  The 
aft  section  of  the  simulator,  the  area  formerly  occupied  by  fuel  cells,  now  contains  the  microprocessor 
racks  which  encompass  the  Advanced  Simulator  Technology  interface.  Figure  1  shows  the  F-16 
simulator  system. 

Computer  Complex.  The  simulator  is  connected  to  a  scries  of  large  and  small  computer 
systems.  This  computer  complex  includes  five  Gould  scries  32/7780,  one  Gould  concept  32/8780,  two 
PDF  11/34,  three  PDF  11/35,  and  two  Silicon  Graphics  Iris  3100  Computer  Aided  Design  stations. 

Experimenter’s  Console.  The  experimenter’s  console  was  located  approximately  twelve  feet 
from  the  simulator.  It  includes  an  intercom  system  which  allows  communication  to  and  from  the  pilot. 
On  the  console  are  displays  which  replicate  the  visual  scene,  HUD,  Data  Entry  Display  and  MFDs. 

The  evaluator  uses  these  displays  to  observe  and  monitor  the  pilot’s  performance,  and  to  start,  stop  and 
reset  the  simulation  as  necessary. 

Voice  (Bogey)  Warning.  A  "bogey"  is  defined  as  an  enemy  aircraft.  Voice  warnings  of  the 
bogey  and  the  bogey’s  clock  position  were  recorded  on  an  Amiga  micro  computer  by  a  female 
employee  of  the  CSEF.  The  employee,  who  had  a  distinctive  and  mature  mid-western  voice,  presented 
messages  in  a  formal  and  impersonal  manner.  The  Amiga  used  a  high  speed  voice  digitizer  (Future 
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Sounds),  with  a  sampling  rate  of  10,000  samples  per  second,  to  convert  the  messages  from  analog  to 
digital  format.  The  Amiga  was  then  connected  to  the  main  frame  computers  using  an  RS-232  interface, 
and  transmitted  the  messages  to  the  pilot’s  head  set  (an  ASTROCOM  model  number  20680  with 
MX-2508/a/c  pads)  through  the  intercom  channel. 


DESIGN 

All  performance  data,  subjective  workload  data  and  situation  awareness  data  were  analyzed 
using  a  repeated  measures  Analysis  of  Variance  (ANOVA).  The  CSEF  questionnaire  data  were 
analyzed  using  a  Chi-square.  The  independent  variables  were  pitch  scale  (3)  and  pitch  number  location 
(2).  The  following  is  a  description  of  each. 

Block  40  F-16.  Figure  2  shows  the  current  F-16  Block  40  HUD  (excluding  data  blocks)  with  a 
longer  horizon  line.  Pitch  bars  were  five  degrees  apart,  and  the  HUD  displayed  20  degrees  of  vertical 
Field  Of  View  (FOV).  Horizontally,  the  HUD  provided  a  FOV  of  30  degrees.  Figure  3  is  identical  to 
Figure  2  except  the  pitch  numbers  were  presented  above  the  outer  end  of  the  pitch  ladders. 

Partially  Articulated  Pitch  Scale.  Figure  4  shows  the  sloped  (articulated)  pitch  bars  below  the 
horizon  line  which  funnel  to  the  horizon.  Similar  to  the  previous  HUD,  the  horizon  line  was  longer, 
pitch  bars  were  five  degrees  apart,  the  vertical  FOV  was  20  degrees  and  the  horizontal  FOV  was  30 
degrees.  The  fourth  configuration  is  shown  in  Figure  S.  In  this  display,  the  pitch  numbers  were  above 
the  outer  end  of  the  pitch  ladders. 

Fully  Articulated  Pitch  Scale.  Figure  6  shows  the  articulated  pitch  bars  above  and  below  the 
horizon  line.  In  both  cases,  the  bars  funnel  to  the  horizon.  As  in  the  other  pitch  ladder  formats,  the 
horizon  line  extended  the  entire  width  of  the  HUD.  Pitch  bars  were  five  degrees  apart,  the  vertical 
FOV  was  20  degrees  and  the  horizontal  FCV  was  30  degrees.  Figure  7  shows  the  same  HUD  with  pitch 
numbers  above  the  outer  end  of  the  pitch  ladders. 

Number  Configuration.  The  effects  of  pitch  indicator  location  were  also  evaluated.  Two 
locations  were  manipulated:  pitch  numbers  at  the  outer  end  and  above  the  pitch  ladders  as  shown  in 
Figures  3, 5  and  7  and  pitch  numbers  at  the  outer  ends  of  the  pitch  ladder  (shown  in  Figures  2, 4  and  6). 

Each  of  the  sbe  configurations  had  a  horizon  line  that  extended  the  entire  width  of  the  HUD, 
but  did  not  occlude  the  vertical  altimeter  and  airspeed  scales  or  the  associated  data  blocks.  Five  levels 
of  pitch  angle  (0,  IS,  -IS,  SS,  -SS)  and  four  levels  of  roll  angle  (0, 4S,  135, 180)  were  used.  The  order  of 
presentation  for  the  recoveries  and  display  configurations  was  counterbalanced. 

Two  types  of  data  were  collected;  objective  and  subjective.  The  dependent  variables  were 
reaction  time,  recovery  time,  correct  response  (nose  low  only)  and  altitude  gain/loss.  Reaction  time  was 
defined  as  the  time  from  the  onset  of  the  HUD  to  the  first  stick  input.  Recovery  time  also  began  with 
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FIOUrHE  2.  BLOCK  40  PfTCM  LADDER  FORMAT  FIOURE  2  BLOCK  40  PITCH  LADDER  FORMAT 

WITH  PfTCH  NUMBERS  ON  THE  END  WTTHPfTCH  NUMBERS  ABOVE 


FIQURES  FUUYARnCULATED  FORMAT  WITH  FIOURE  7,  FULLY  ARTICULATED  FORMAT  WITH 

PITCH  NUMBERS  ATTHE  END.  PITCH  NUMBERS  ABOVE. 
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the  onset  of  the  HUD,  but  it  continued  until  the  pilot  recovered  the  simulator  to  + 10/- 10  degrees  of  ' 
pitch  and  + 10/-10  degrees  of  roll.  A  correct  response  was  defined  as  an  mitial  stick  movement  which 
caused  the  aircraft  to  take  the  shortest  route  to  the  upright  and  wings  level  position.  Also,  the  pilot 
should  not  put  G  forces  on  the  aircraft  until  it  is  within  60  degrees  of  bank.  Altitude  gain/loss  was  the 
amount  of  altitude  gained  or  lost  during  the  recovery. 

The  subjective  dependent  variables  included  Subjective  Workload  Assessment  Technique 
(SWAT),  Subjective  Workload  Dominance  (SWORD)  and  Situation  Awareness  (SA)  scores.  The 
following  section  describes  each  of  these  subjective  tools. 

SWAT.  SWAT  has  been  used  in  the  simulation  environment  for  several  years  to  assess  pilot 
mental  workload,  and  is  comprised  of  two  phases:  a  scale  development  phase  and  an  event  scoring 
phase.  For  a  thorough  description  of  the  two  phases,  refer  to  the  SWAT  User’s  Guide  (Reid  et  al., 
1989).  Once  all  event  scores  are  collected,  rescaled  values  are  assigned  to  the  pilot’s  SWAT  rating  (see 
Table  1).  The  result  is  a  workload  value  ranging  from  0  to  100.  With  these  values,  a  mean  SWAT  score 


Table  1.  Rescaled  SWAT  Values. 


can  be  computed  which  indicates  the  amount  of  workload  mduced  by  flying  with  a  particular  HUD, 
Higher  mean  scores  mdicate  higher  workload.  A  3  X  2  X  2  repeated  measures  design  was  used.  The  in¬ 
dependent  variables  were  three  pitch  scale  formats,  two  pitch  number  locations  and  two  types  of  task 
loading.  The  HUD  configurations  were  previously  described.  The  first  type  of  task  load  dealt  with  the 
amount  of  difficulty  the  pilot  had  in  recognizing  or  recovering  from  the  unusual  attitude.  An  easy  at¬ 
titude  was  one  that  was  simple  to  recognize  or  recover  from,  while  a  difficult  attitude  was  harder  to 
recognize  or  recover  from.  A  post  hoc  definition  of  attitude  difficulty  was  made,  and  Table  2  shows 
how  the  attitudes  were  categorized.  The  second  type  of  task  load  dealt  with  the  presence  or  absence  of 


Ttbit  3.  Euy  ind  Dllficult  AtUtudM  From  Which  to  Recover 
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a  bogey.  During  the  pre-flight  briefing,  pilots  were  told  that  a  normal  trial  consisted  of  a  miiision  flown 
in  the  weather  in  which  the  pilot  becomes  disoriented.  Then,  using  the  HUD,  he  was  to  recover  the 
aircraft  to  wings  level.  On  18  random  trials  the  bogey  (voice)  warning  was  presented.  The  pilot  was  in¬ 
structed  to  execute  an  appropriate  recovery,  and  then  once  the  aircraft  was  at  wings  level,  locate  the 
bogey.  ,SWAT  event  scores  were  collected  after  each  bogey  trial  and  the  trial  that  succeeded  the  bogey 
trial  (which  had  the  same  initial  attitude).  During  data  analysis,  a  direct  comparison  of  pilot  workload 
was  possible  for  bogey/no  bogey  conditions.  The  dependent  variable  was  the  pilot’s  SWAT  score. 


SWORD.  SWORD  is  a  subjective  workload  tool  developed  by  Dr.  M.  A.  Vidulich  at 
Armstrong  Aerospace  Medical  Research  Laboratory  (AAMRL).  With  SWORD,  pilots  make  relative 
judgements  on  a  17  point  scale  comparing  displays  as  shown  in  Figure  8.  Making  SWORD  ratings  is 
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FIQURE  i.  RELATIVE  WORKLOAD  JUDOEMENTS  EXAMPLE  SHEET. 
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very  simple  and  is  accomplished  at  the  end  of  simulation.  After  numbers  are  assigned  to  the  relative 
ratings,  a  computer  is  used  to  calculate  the  geometric  mean  SWORD  score  for  each  display.  The 
display  which  induced  the  least  amount  of  workload  during  flight  will  have  a  lower  mean  score,  while 
the  highest  mean  score  indicates  the  display  which  induced  the  highest  workload.  A  3  X  2  X  2  (pitch 
scale  X  number  location  X  task  load)  repeated  measures  design  was  employed.  For  SWORD,  task  load 
was  defined  as  the  amount  of  difficulty  the  pilot  had  in  recognizing  or  recovering  from  the  unusual 
attitude.  An  easy  attitude  (-IS  degrees  pitch  and  60  degrees  roll)  was  one  that  was  simple  to  recognize 
or  recover  from,  while  a  difficult  attitude  (-SS  degrees  pitch  and  135  degrees  roll)  was  more  difficult  to 
recognize  or  recover  from.  SWORD  ratings  were  based  upon  the  workload  induced  when  the  HUD 
was  in  one  of  these  two  attitudes.  The  dependent  variable  was  the  pilot’s  SWORD  score. 

Situation  Awareness.  A  3  X  2  X  2  (pitch  scale  X  number  location  X  task  load)  repeated 
measures  ANOVA  was  used  for  the  analysis  of  SA  data.  Situation  awareness  was  assessed  using  a 
technique  developed  by  Major  M.  Fracker  at  AAMRL.  The  SA  rating  instructions  and  scale,  are  shown 
in  Figure  9.  Basically  a  statement  is  made  concerning  the  pilot’s  SA  when  using  a  particular  display, 
then  during  simulatipn,  the  pilot  gives  a  rating  based  upon  the  amount  that  he  agrees  or  disagrees  with 
that  statement.  In  this  case  the  statement  was  T  experienced  no  confusion  with  this  HUD 
configuration,  and  was  easily  able  to  recover  to  straight  and  level  flight."  Each  pilot’s  SA  rating  was 
recorded  following  random  trials.  These  trials  were  the  same  trials  used  to  collect  SWAT  event  scores. 
The  ordinal  level  ratmgs  given  by  pilots  were  converted  to  equivalent  interval  scale  values  (see  Figure 
9)  using  the  Training  and  Human  Factors  Research  on  Military  Systems:  Questionnaire  Construction 
Manual  {^tmy  Manual  P-71-1).  The  interval  level  scale  ranged  from  -2.76  through  2.77.  The  scale  value 
that  resulted  was  the  pilot's  situation  awareness  score  and  was  used  as  a  dependent  variable. 


PROCEDURE 


Preflight  Briefing 

The  pilots  received  a  standardized  briefing  describing  the  procedures  to  use  during 
simulation,  the  description  of  each  pitch  ladder  and  number  format  and  SWAT  qard  sort  instructions. 
Additional  instructions  included  SA  data  collection  procedures,  which  were  read  to  the  pilot  (refer  to 
Figure  9). 

Pilots  were  instructed  to  execute  each  recovery  as  they  would  during  a  normal  F-16  flight 
while  considering  realistic  positive  and  negative  ’g'  and  energy  management.  For  nose  high  attitudes, 
the  pilot  was  told  to  roll  to  90  degrees,  let  the  nose  of  the  aircraft  fall  and  then  roll  to  wings  level  near 
the  horizon.  For  nose  low  conditions,  pilots  were  instructed  to  roll  to  wings  level  and  pull  to  the 
horizon.  Pilots  were  briefed  that  they  would  fly  23  trials  with  each  pitch  scale  and  number 
configuration  fur  a  subtotal  of  138  trials;  18  of  which  involved  a  bogey  warning.  The  bogey  warning 
was  a  verbal  warning  of  a  "bogey"  and  a  specific  clock  position  (i.e.  BOGEY  4  O’CLOCK)  and 
was  intended  to  increase  pilot  workload.  Following  the  preflight  briefing,  pilots  were  given  time  to 
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SUBJECTIVE  PITCH  LADDER  RATING  INSTRUCTIONS 


In  this  svaluatlon,  ws  are  intsrastsd  In  how  wall  diffarsnt  pitch  Isddar  configurations  affect  your  attituda 
awaranass.  A  poor  design  may  misisad  you  to  Initially  pitch  ths  wrong  way,  or  you  may  pitch  the  right  way 
but  ovsrshoot  or  undershoot,  After  certain  trials,  ws  will  ash  you  to  evaluate  the  pitoh  ladder  configuration 
you  Just  used.  You  will  be  asked  to  respond  to  ths  following  etatsmint:  *1  experienced  no  confusion  with  this 
pitch  ladder  configuration  and  was  easily  abis  to  recover  to  straight  and  level  flight.*  Simply  Indloate  the 
extent  to  which  you  agree  with  the  statement  by  giving  a  number  from  1  to  B.  Use  the  following  scale. 


S-DooldedlyAQREE 

5-  Substantially  AQREE 
7  •  Moderately  AGREE 

6 - Perhaps  AQREE 
S- BORDERLINE 

4 -Perhaps  DISAGREE 
3 -Moderately  DISAGREE 
2 -Substantially  DISAGREE 
1  -  Decidedly  DISAGREE 


Scale  Values  for  Descriptors: 

9  •  Decidedly  agree  2.77 


6  •  Sustantially  agree . ,...2,10 

7  -  Moderately  agree . 1.47 

0  •  Perhaps  agree . 0.02 

S  •  Borderline . 0.00 

4  -  Perhaps  disagree . . . ^),43 

3  -  Moderately  disagree . -1 .38 

2  -  Substantially  disagree . -2.17 

1  -  Decidedly  disagree . -2.78 


Source  for  scale  dssorlptors  and  scale  values:  Babbit,  B.  A.  &  Nystrom,  C.  0.  (1B88).  Training  and  human 
factors  research  cn  military  systems;  Questionnaire  oonstruotlon  manual  (Army  Manual  P-71-1|.  Port  Hood, 
TX:  Army  Research  Institute  for  the  Behavioral  Sclenoes. 


FIGURE  g.  8A  RATING  INSTRUCTIONS  AND  SCALE. 
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perform  the  SWAT  card  sort  (30-60  minutes). 

Training 

Pilots  were  trained  on  each  of  the  configurations  before  any  data  were  collected.  Prior  to 
flymg  each  new  configuration,  one  free-flight  practice  trial  was  flown  until  the  pilot  indicated  that  he 
felt  comfortable  with  the  display  (maximum  of  five  minutes). 

Trial  Procedure 

The  HUD  was  blanked  out  and  the  Horizontal  Situation  Indicator  (HSI)  and  standby  attitude 
indicator  were  frozen  while  an  autopilot  system  flew  the  simulator  mto  an  unusual  attitude.  The 
Attitude  Director  Indicator  (ADI)  was  removed  for  the  entire  simulation.  Once  the  autopilot  reached 
the  unusual  attitude,  the  pilot  received  the  initial  course  heading  and  pulled  the  Display  Management 
Switch  (DMS),  located  on  the  stick,  to  the  aft  position.  Activation  of  the  DMS  switch  initiated  the  trial 
which  made  the  HUD  reappear  and  released  the  HSI  and  standby  attitude  mdicatcr.  The  pilot  flew  the 
recovery  to  wings  level.  When  the  recovery  was  complete  ( + 10/-10  degrees  of  pitch  and  + 10/- 10 
degrees  of  roll)  the  pilot  pulled  the  trigger  which  ended  the  trial  and  reset  the  HUD,  HSI  and  standby 
attitude  indicator.  Airspeed,  altitude,  vertical  velocity  and  angle  of  attack  were  initially  the  same  for 
each  trial.  For  trials  in  which  a  bogey  warning  was  presented,  the  warning  was  given  0.3  seconds  after 
the  pilot  activated  the  Display  Management  Switch  (DMS)  to  begin  the  trial.  If  ilots  were  instructed  to 
remember  the  clock  position,  perform  a  recovery  as  they  would  in  an  actual  bogey  situation,  and  then, 
relative  to  their  recovery  heading,  tell  the  evaluator  where  the  bogey  was  located.  For  simulation 
purposes,  the  bogey  did  not  move,  however,  while  executing  the  recovery  the  pilot’s  heading  may  have 
changed,  thus  relative  to  the  new  heading,  the  bogey  may  have  a  new  clock  position.  For  the  bogey 
trials  and  the  trial  immediately  following  it,  the  initial  attitudes  were  identical.  After  these  pairs  of 
trials,  SWAT  event  scores  and  situation  awareness  (SA)  ratings  were  collected.  Pilots  were  instructed 
to  give  SWAT  scores  and  SA  ratings  via  the  intercom. 

Debrieflng 

The  evaluator  used  the  debriefing  to  discuss  observations  made  during  simulation  and  to 
collect  the  SWORD  data.  There  were  two  parts  to  the  SWORD  ratings;  one  part  Involved  SWORD 
ratings  based  upon  easy  attitude  deviations  (-15  degrees  pitch  and  60  degrees  roll)  and  the  other  part 
was  based  upon  difficult  attitude  deviations  (-55  degrees  pitch  and  135  degrees  roll). 

RESULTS 

Objective  Data 

Two  independent  variables  were  examined  as  a  function  of  reaction  time,  recovery  time, 
correct  response  (nose  low  only)  and  altitude  galn/ioss  in  a  3  X  2  repeated  measures  ANOVA.  Results 
are  presented  by  attitude  condition  i.o.  nose  low  or  nose  high.  "Nose  low"  consisted  of  initial  attitudes 
with  degrees  of  pitch  less  than  zero,  while  "nose  high"  wore  initial  attitudes  with  degrees  of  pitch 


1.1 


greater  than  or  equal  to  zero. 


The  cockpit  environment  associated  with  the  present  simulation  (and  with  the  aircraft)  is  so 
diverse  and  dynamic  that  attempting  to  restrict  the  pilot’s  focus  to  one  specific  instrument  is  virtually 
impossible.  Thus,  while  the  objective  of  this  study  was  to  evaluate  pitch  ladder  formats,  most  attitude 
related  instruments  were  incorporated,  resulting  in  a  more  complete  mission  scenario.  This,  in  turn, 
decreased  the  level  of  experimenter  control  over  the  pilot’s  decisions  and  actions  during  the  task 
performance.  To  avoid  overlooking  any  significant  effects  between  the  different  configurations  during 
the  evaluation,  it  seemed  appropriate  to  assume  a  liberal  stand  in  rejecting  the  null  hypothesis  by 
selecting  a  confidence  level  (p  value)  of  less  than,  or  equal  to  0.10. 

Nose  Low  Attitudes 

Reaction  Time.  An  ANOVA  revealed  no  significant  bteraction  between  the  pitch  number 
location  and  pitch  scale  format  for  the  mean  reaction  times,  F(2,22)  -•  .02,  p  « .98.  The  main  effect  of 
pitch  scale  format  was  significant,  F(2,22)  ■  1.06,  p  .08.  An  inspection  of  Figure  10  suggests  that 
recoveries  performed  using  the  partially  articulated  format  (0.37  seconds)  resulted  in  the  quickest 
mean  reaction  time,  followed  by  the  fully  articulated  configuration  (0.46  seconds)  and  the  Block  40 
configuration  (0.49  seconds).  Significance  was  nut  found  for  the  main  effect  of  number,  F(l,ll) » 1.07, 
P-.32. 

e 


WCHlAOOSRFOflMAT 

PKJUMIO.  MEAN  nSACTION  TIMS  AS  A  FUNCTION  OF  PITCH  LAOOER  FORMAT  (NOSE  LOW). 

Recovery  Time.  An  examination  of  the  three  pitch  ladder  format  X  two  pitch  number  location 
interaction  (Figure  11)  showed  significance,  with  F(2,22)»2.59,  p-.lO.  The  partially  articulated  pitch 
scale  with  pitch  numbers  presented  above  the  pitch  ladders  resulted  in  the  best  mean  recovery  time 
(4.20  seconds).  The  longest  recovery  times  occurred  with  the  fully  articulated  format  with  pitch 
numbers  at  the  ends  of  the  pitch  ladder  (5.0  seconds).  Neither  the  main  effect  of  pitch  scale 
(F(2,22)  -  2.59,  p  •»  .26)  nor  pitch  number  location  (F(l,ll)  -  .41,  p  *  .54)  were  found  to  be  significant. 
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NUMBiRCONFIOUFUTION 


PIgur*  1 1 .  MEAN  RECOVERV  TIMS  A8  A  PUNOTION  OF  NUMBER 

CONFiaURAT)ON  AND  PtTCH  LAOOER  FORMAT  (NOSE  LOW). 

Correct  Response.  For  nose  low  conditions  only,  the  percentage  of  correct  initial  stick  inputs 
was  determined  as  a  function  of  pitch  scale  format  and  number  configuration,  From  Figure  1'2,  it  is 
obvious  that  flying  with  one  configuration  was  essentially  the  same  as  flying  with  all  configurations. 

The  percentage  of  correct  responses  for  the  four  articulated  pitch  scales  was  nearly  identical 
(either  70  or  72  percent).  However,  with  the  F<16  Block  40  configurations,  the  percentages  ranged  from 
64  to  76  percent.  These  two  configurations  were  exactly  the  same  except  for  pitch  number  location. 

This  result  indicates  that  the  pitots  were  basing  their  first  stick  input  decision  on  the  cue  provided  by 
number  location. 


END  ABOVE 

NUMBER  OONFKIUnATION 


FIQURE  1 3.  CORRECT  RESPONSE  PERCENTAaE  AS  A  FUNCTION  OP 
NUMRER  CONFiaURAnON  AND  PtTCH  lAOOER  FORMAT. 


Altitude  Loss.  Another  dependent  variable  used  in  this  evaluation  was  altitude  loss  during  the 
recovery.  The  interaction  between  pitch  scale  and  pitch  number  location  was  not  significant, 

F(2,22)  - 1.88,  p  -  .18.  Furthermore,  neither  the :  Kain  effect  of  pitch  scale,  F(2,22)  - 1.06,  p  =  .36,  nor 
the  main  effect  of  pitch  number  location,  F(l,ll)  »*  .02,  p“  .89,  were  significant. 

Nose  High  Attitudes 

Table  3  shows  that  none  of  the  pitch  scale  format  effects,  pitch  number  location  effects  or  their 
interactions  were  significant  for  any  of  the  dependent  variables:  reaction  time,  recovery  time  or  altitude 
gained. 


Table  3.  ANOVA  Table  for  Phase  1  Nose  High  Attitudes. 
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Subjective  Data 


The  following  describes  the  SWAT,  SWORD,  SA  and  CSEF  questionnaire  results.  For 
SWAT,  the  independent  variables  included  three  pitcJi  scale  formats  and  two  types  of  task  loading; 
bogey/no  bogey  conditions  and  easy/difficult  attitudes  from  which  to  recover.  For  SWORD,  three  pitch 
scale  formats,  two  number  locations  and  two  task  loading  levels  were  manipulated,  and  for  SA,  three 
pitch  scale  formats,  two  number  configurations  and  two  task  loadmg  levels  were  manipulated. 

SWAT 


Pilots  can  fly  almost  any  display  under  normal  flight  conditions.  However,  given  a  situation 
where  workload  is  significantly  higher,  the  results  of  a  poor  HUD  may  be  disastrous.  For  this  reason, 
SWAT  ratings  were  used  to  provide  a  measure  of  pilot  workload  for  each  of  the  pitch  ladder  formats. 
The  ANOVA  results  for  the  various  task  loading  levels  will  now  be  described. 

The  tluee-way  Interaction  of  pitch  scale  format,  attitude  difficulty  and  bogey  condition  was  not 
significant  for  F(2,22)  ■  .75,  p  ■■  .48.  Figure  13  shows  the  means  for  attitude  difficultly  and  pitch  ladder 
format.  This  interaction  was  significant,  F(2,22)  ■>  9.12,  p»  .001.  For  difficult  attitudes  the  partially 
articulated  pitch  scale  (mean  SWAT  score  ■  13.8)  induced  lower  workload  than  the  other  two  pitch 
scale  formats.  When  recovering  from  easy  attitudes,  lower  workloads  were  induced  flying  the  fully 
articulated  format  (mean  SWAT  score  -  8.4).  Also,  the  partially  articulated  piToh  ladder  required 


PITCH  LADDER  FORMAT 


n  BLOCK  40 

□  PARTIAL 

□  FULL 


FIGURE  13.  MEAN  SWAT  SCORE  AS  A  FUNCTION  OF  TASK  LOAD 
(DIFFICULTY)  AND  PITCH  LADDER  FORMAT. 
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essentially  the  same  amount  of  workload  regardless  of  attitude.  Figure  14  illustrates  the  main  effect  of 
attitude  difficulty.  Clearly,  the  attitudes  that  were  dlfHcult  to  recover  from  induced  significantly  more 
workload  (mean  SWAT  score  =  17.9)  than  the  easier  recovery  attitudes  (mean  SWAT  score  =  11.7), 
F(l,ll)  =  18.66,  p  =  0.001. 
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TASK  LOAD 

FIGURE  1 4,  MEAN  SWAT  SCORE  AS  A  FUNCTION  OF  TASK  LOAD  (DIFFICULTY), 

Other  analysis  examined  the  interaction  of  pitch  ladder  format  and  bogey  conditions.  This 
interaction  was  not  significant,  F(2,22)  ■  ,21,  p  « .81  The  main  effect  of  the  bogey  warning  was 
significant,  F(l,ll) «  57.27,  p  ••  .0001.  Bogey  warning  conditions  yielded  a  mean  SWAT  score  of  22.2, 
while  no  bogey  conditions  had  a  mean  SWAT  score  of  6.99.  Figure  15  shows  the  SWAT  scores  for 
bogey  and  no  bogey  conditions.  Finally,  the  main  effect  of  pitch  ladder  format  was  not  significant, 
F(2,22)-l,59,p-.23. 


NO  BOGEY  BOGEY 

TASK  LOAD  (BOGEY) 

FIGURE  18.  MEAN  SWAT  SCORES  AS  A  FUNCTION  OF 
TASK  LOAD  (BOGEY  CONDITION). 
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SWORD 


Following  simulation,  pilots  performed  two  SWORD  evaluations;  one  for  relatively  easy 
attitude  adjustments  (-15  degrees  pitch  and  60  degrees  roll)  and  one  for  relatively  difficult  attitude 
adjustments  (-55  degrees  pitch  and  135  degrees  roll).  Three  pitch  ladder  formats  and  tv/o  number 
configurations  were  rated. 

The  three-way  interaction  of  pitch  ladder  format  X  number  location  X  attitude  showed  no 
significance,  F(2,20) » 1.44,  p  «■  .26.  An  examination  of  the  pitch  scale  X  pitch  number  interaction, 
F(2,20)  ■■  1.80,  p  B  .19,  yielded  no  significant  differences  either  .The  pitch  ladder  X  attitude  interaction 
showed  significance,  F(2,20)  -3.95,  p-  .04,  and  an  examination  of  the  means,  shown  in  Figure  16, 
suggests  that  for  difficult  adjustments,  the  partially  articulated  pitch  scale  (mean  SWORD  rating  - 
.0809)  and  the  fully  articulated  pitch  scale  (.0898)  induced  significantly  less  workload  then 


TASK  LOAD  PFFICULTV) 


n  ULOCKAO 

□  PAmiAL 

□  FULL 


FIQURE  I  a.  MEAN  8WORO  SCORE  AS  A  FUNCTION  OF  TASK  LOAD 
(DIFFICULTO  FOR  ALL  PITCH  lADDER  FORMATS. 


Block  40  (.3291).  The  impact  that  attitude  deviation  and  number  configuration  had  on  mean  SWORD 
ratings  was  also  examined,  and  the  two  way  interaction  was  not  significant,  F(l,10)  -  .02,  p  -  .89.  The 
main  effect  of  pitch  ladder  was  significant,  F(2,20)  -  31.05,  p  -  .001,  with  the  means  shown  in  Figure  17. 
The  partially  articulated  format  induced  the  lowest  amount  of  workload  (mean  SWORD  rating  - 
.0950),  followed  by  fully  articulated  (.0979)  and  Block  40  (.3069).  The  effect  of  pitch  number  configura¬ 
tion  was  also  examined,  The  workload  differences  induced  by  pitch  number  location  were  not  sig¬ 
nificant,  F(l,10)  -  .46,  p  “  .51. 
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nrCHUOOERFOflMAT 


MURE  17.  MEAN  SWORO  SCORE  AS  A  FUNCTION  OF 
PITCH  LADDER  FORMAT. 


Situation  Awareness 

Each  pilot’s  SA  was  recorded  following  selected  random  trials.  The  pitch  ladder  format  X 
pitch  number  location  X  bogey  interaction  was  not  statistically  significant  for  F(2,22)  ■■  .04,  p  » .96. 
Likewise,  none  of  the  two  way  interactions  were  significant;  pitch  ladder  X  pitch  number, 

F(2,22)  •  1.32,  p  u!  .29,  pitch  ladder  X  bogey,  F(2,22)  « .11,  p  -  .89  and  pitch  number  X  bogey, 

F(l,ll)  .56,  p  >  .47  The  main  effect  of  number  location  was  not  significant,  F(l,ll)  ■  .36,  p  -  .56. 
Figure  18  shows  the  main  effect  of  pitch  scale  format,  F(2,22)  «•  5.47,  p  ■■  .01,  which  was  statistically 
significant.  An  examination  of  the  means  suggest  that  pilot  SA  was  better  when  flying  with  the  fully 
articulated  pitch  scale  format  (mean  SA  score  2.04)  and  the  partially  articulated  format  (1.98)  then 
with  the  F-16  Block  40  configuration  (1.55). 


PITCH  LADDER  FORMAT 

FIGURE  1«.  MEAN  SITUATION  AWARENESS  SCORE  AS  A  FUNCTION 
OF  PITCH  LADDER  FORMAT. 
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Similar  to  the  bogey  results  from  the  SWAT  data,  the  SA  main  effect  of  bogey  was  significant, 
F(l,ll)  -34.82,  p  -  .0001.  Figure  19  suggests  that  the  bogey  warning  trials  significantly  reduced  pilot 
situation  awareness.  Other  SA  results  are  discussed  in  Appendix  A. 


TASK  LOAD  LEVEL 

mURE  1«.  MEAN  8ITUATX3N  AV^ARENESS  SCORE  AS  A 
HUNCnON  OR  TASK  LOAD  (BOQEY). 

CSEF  Questionnaire 

At  the  conclusion  of  Phase  I  simulation,  part  one  of  a  CSEF  questionnaire  was  given  to  the  . 
pilots.  The  questionnaire  is  shown  in  Appendix  B.  ludiwdual  Chi-Square  tests  were  performed  on  the 
questionnaire  data.  For  each  question,  response  frequency  was  indicated,  and  if  significant,  graphed. 

Pitch  Ladder.  As  shown  in  Figure  20,  the  articulated  pitch  scales  were  preferred  over  the 
Block  40  pitch  scale.  Six  pilots  preferred  the  partially  and  six  preferred  the  fully  articulated  pitch 
scale.s.  The  frequency  analysis  indicated  that  this  preference  was  significant,  >•  6.0,  p  <  .05. 


PILOT  RESPONSE 


FIGURE  ao.  NUMBER  OF  PILOTS  WHO  PREFERRED  EACH 
OF  THE  THREE  TYPES  OP  PITCH  LADDERS. 
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Pitch  Number  Location.  The  Chi-Square  analysis  of  pitch  number  location  yielded  no 
significant  differences  between  either  of  the  alternatives  (x^  1.43,  p  >  .05), 

Flight  Path  Marker.  Articulated  pitch  bars  pose  a  special  problem  when  trying  to  determine 
degrees  of  pitch.  If  the  flight  path  marker  is  located  between  the  upper  ends  of  the  pitch  ladders,  then 
should  the  pilot  read  degrees  of  pitch  from  that  upper  end?  What  if  the  flight  path  marker  is  located 
between  the  lower  ends  of  the  pitch  bars,  then  does  the  pilot  read  degrees  of  pitch  from  that  location? 
Pilots  were  asked  this  question  but  no  significant  difference  existed  between  these  two  options,  ■■ 
1.34,  p  >  .05. 

Spatial  Disorientation.  Seven  of  the  12  pilots  had  experienced  spatial  disorientation  using  the 
HUD.  A  Chi-Square  of  the  frequencies  showed  no  significant  differences,);^  -  .33,  p  >  .05. 

Articulated  Pitch  Ladders,  The  following  section  includes  a  discussion  of  pilot  responses  to 
two,  two-part  questions  concerning  articulated  pitch  bars.  The  fully  articulated  pitch  bar  format  helped 
pilots  distinguish  between  nose  up  and  nose  down  attitudes  in  eight  out  of  12  cases,  however,  this  result 
was  not  significant,);^  1.33,  p  >.05.  With  the  partially  articulated  format,  all  12  pilots  were  better 
able  to  distinguish  between  nose  high  and  nose  low  attitudes,  **  12.0,  p  <  .05,  (see  Figure  21). 
Although  articulation  was  very  useful  in  distinguishing  between  nose  low  and  nose  high  attitudes,  the 
same  was  not  true  in  determining  roll.  Seven  pilots  stated  that  both  the  fully  0;^  «  .33,  p  >  .05)  and 
.  partially  (x^ »  .33,  p  >  .05)  articulated  pitch  scales  were  helpful  in  determining  roll,  but  these  results 
were  not  significant. 
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Tails  at  the  Ends  of  the  Pitch  Bars.  The  tails  or  "tic  marks"  at  the  ends  of  the  pitch  bars  can 
either  point  to  the  ground  (nndir)  or  the  horizon.  When  asked  their  preference,  all  12  pilots  favored 
horizon  pointing  tails  and  the  freqencies  are  shown  in  Figure  22,  »  12,  p  <  .05).  The  results  were 

significant. 
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NADIR  P0INTIM3 


HORIZON  POINTING 


PILOT  RESPONSE 

FIGURE  82.  FREQUENCY  OF  PILOTS  WHO  PREFER  NADIR 
AND  HORIZON  POINTING  TAILS. 

SUMMARY 

Generally,  the  performance  results  indicated  that  the  partially  articulated  format  was  the  best 
HUD  configuration.  Subjective  data  tended  to  support  this  finding,  particularly  under  high  workload 
conditions.  The  performance  and  subjective  results  did  not  show  that  one  pitch  number  location  was 
better  than  the  other. 


-EHASEJI 

METHOD 

SUBJECTS 

Pilots  who  participated  in  Phase  I  also  participated  in  Phase  11. 

APPARATUS 

The  facility,  F-16  simulator,  computer  complex  and  experimenter’s  console  were  the  same  as 
those  used  in  Phase  I  and  arc  described  in  the  Phase  I  Method  section, 

DESIGN 

Performance  data  were  analyzed  using  a  within  subjects  ANOVA,  and  a  Chi-Square  was  used 
to  analyze  the  questionnaire  data.  The  only  independent  variable  was  horizon  line  length.  Two  horizon 
lines  were  evaluated.  One  horizon  line  was  the  F*16  Block  40  horizon  line  (Figure  23),  while  the  other 
extended  the  entire  width  of  the  HUD  as  shown  in  Figure  24.  The  longer  horizon  did  not  occlude  the 
vertical  altimeter  and  airspeed  scales  or  the  associated  data  blocks.  All  other  symbology  was  F<16 
Block  40  symbology.  Pitch  and  roll  conditions  were  the  same  as  those  used  in  Phase  I.  This  design 
resulted  in  40  trials.  The  order  of  presentation  for  the  recoveries  and  display  configurations  was 
counterbalanced.  Dependent  variables  were  reaction  time,  recovery  time  and  altitude  gain/loss  as 
deHned  in  the  Phase  I  Design  section. 

PROCEDURE 

Preflight  Briefing 

The  preflight  briefing  for  Phase  11  followed  the  Phase  I  debriof.  All  pilots  were  given  the 
standardized  briefing.  The  briefing  included  a  description  of  the  procedures  to  follow  during 
simulation  and  a  description  of  each  format  to  be  evaluated.  Pilots  were  reminded  to  recover  from  the 
unusual  attitude  using  the  procedures  briefed  during  Phase  I. 

Training 

Training  for  Phase  II  was  the  same  as  Phase  I  training.  Pilots  were  given  the  opportunity  to  fly 
each  format  before  any  data  were  collected.  Prior  to  flying  each  new  format  a  five-minute  free  flight 
practice  period  was  provided  to  the  pilot.  After  the  training  period,  the  simulation  began. 


.22 


FIGURE  23.  HEAD-UP  DISPLAY  WITH  NORMAL  HORIZON  LINE. 


FIGURE  24.  HEAD-UP  DISPLAY  WITH  EXTENDED  HORIZON  LINE. 
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Trial  Procedures 


Similar  to  Phase  I,  pilots  were  required  to  recover  from  unusual  attitudes.  All  trial  procedures 
were  ‘.he  same  except  that  the  measures  of  workload  and  situation  awareness  were  not  used  in  Phase  11. 

Debriefing 

Debriefings  consisted  of  an  informal  interview  in  which  the  pilots  discussed  any  problems 
encountered  during  simulation  or  with  the  simulator,  and  their  opinion  of  the  displays,  The  evaluation 
concluded  with  the  pilots  completing  a  CSGF  questionnaire. 

RESULTS 

Objective  Data 

During  Phase  II  performance  data  were  collected  on  two  horizon  lines.  Results  are  presented 
by  initial  attitude  condition  (nose  low  or  nose  high).  Statistical  significance  was  not  found  for  any  of  the 
Phase  II  results.  The  ANOVA  results  are  shown  in  Table  4. 

Tabla  4.  ANOVA  Table  for  tha  Phaaa  It  Evaluation 
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Subjective  Data 


The  CSEF  questionoaire  for  Phase  II  addressed  the  issue  of  horizon  line  length  and  the  utility 
of  the  bank  indicator  for  unusual  attitude  recoveries.  Individual  one-way  Chi-Square  tests  were 
performed  on  the  questionnaire  data.  For  each  question,  response  frequency  was  indicated  and,  if 
significant,  graphed,  The  questionnaire  is  shown  in  Appendix  D  and  pilot  comments  are  provided  in 
Appendix  E. 

Hortson  Line.  The  frequency  data  for  horizon  line  preference  are  shown  in  Figure  25.  Eleven 
of  the  12  pilots  thought  the  longer  horizon  line  was  a  good  idea.  This  result  was  significant,  ■■  28.6, 
p<  .05. 


GOOD  IDEA  PREFER  BLOCK  40 

NO  OPINION  DISTRACTING 

PILOT  RESPONSE 

FIGURE  25.  PILOT'S  OPINIONS  OF  THE  LONGER  HORIAON  LINE. 

Bank  Indicators.  At  the  conclusion  of  the  Phase  II  simulation,  pilots  were  given  the 
opportunity  to  fly  with  a  bank  indicator  that  had  hash  marks  at  45  degree  angles  around  the  flight  path 
marker  (FPM).  Then,  pilots  were  asked  their  preference  between  the  present  bank  ir>dicator  and  the 
bank  indicator  around  the  flight  path  marker.  Pilots  favored  having  the  bank  indicato'  presently  used 
significantly  more  than  the  other  option,  ■■  3.33,  p<  .03.  Frequencies  are  shown  in  Figure  26. 

Pointer  on  the  Bank  Scale.  Pilots  were  asked  their  opinion  of  a  ground  pointer  and  a  sky 
pointer  on  the  bank  scale.  The  frequencies  are  shown  in  Figure  27.  Ten  pilots  favored  the  sky  pointer 
and  two  favored  the  ground  pointer.  The  results  were  significant,  x^  3.33,  p  <  ,05.  The  CSEF 

engineers  were  Interested  in  pilot  opinions  of  the  bank  scale  for  two  reasons;  (1)  to  detertr.ine  what 
role,  if  any,  the  bank  indicator  played  in  unusual  attitude  recoveries  and  (2)  to  determiFic  the  most 
favorable  features  for  a  bank  indicator.  When  recovering  from  unusual  attitudes,  nine  of  the  pilots 
indicated  that  they  did  not  use  the  bank  indicator.  This  was  expected  because  pilots  arc  not  trained  to 
use  the  bank  indicator  for  unusual  attitude  recoveries.  This  result  was  not  significant, x'  ■>  3.0,  p  >  .05. 
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Six  of  the  pilots  felt  that  a  360  degree  bank  indicator  was  a  good  idea,  but  statistical  significance  was 
not  found,  ^  -  6.67,  p  >  .05.  Pilots  were  asked  their  opinion  of  having  a  bank  pointer  pointing  up.  In 
this  case,  significant  differences  were  found;  seven  pilots  felt  that  a  bank  pointer  pointbg  up  was  of 
some  use  at  low  bank  angles,  one  felt  it  was  very  helpful,  two  had  no  opinion  and  two  felt  it  was  of  little 
use  ■>  12.2,  p  <  .05).  Figure  28  shows  the  frequencies  for  each  response.  Finally,  six  pilots  felt  that  a 
bank  pointer  should  be  limited  to  +/•  45  degrees  and  six  felt  it  should  not  Significant  differences  did 
not  exist,  -  0,  p  >  .05. 
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DISCUSSION 

This  section  discusses  both  phases  of  the  present  evaluation.  Two  general  findings  from  the 
results  were:  (1)  the  Block  40  pitch  scale  was  the  worst  format  for  both  nose  high  and  nose  low 
conditions  and  (2)  the  partially  articulated  pitch  scale  was  significantly  better  than  the  other  pitch 
scales  for  nose  low  attitudes.  This  makes  sense  because  the  partially  articulated  pitch  scale  in  a  nose 
low  attitude,  gives  the  pilot  instantaneous  feedback  concerning  the  aircruft‘s  attitude. 

Training  effects  may  account  for  the  quicker  recovery  times  found  flying  the  Block  40  with 
pitch  numbers  at  the  end  (4.66  seconds),  since  F>16  pilots  fly  this  format  every  day.  Due  to  the  pitch 
ladder  symmetry  found  on  the  fully  articulated  HUD,  a  pilot  must  make  a  distinction  between  the  solid 
bar  (nose  high)  and  dashed  bar  (nose  low)  to  determine  whether  he  is  inverted.  However,  with  pitch 
numbers  above  the  pitch  ladder,  a  pilot  is  immediately  aware  of  his  attitude  (upright  or  inverted).  This  . 
explains  the  reason  for  a  quicker  mean  recovery  time  for  the  fully  articulated  pilch  scale  with  pitch 
numbers  above  the  pitch  bars. 

Both  subjective  workload  measures  yielded  essentially  the  same  results.  Articulated  pitch 
scales  were  significantly  better  than  the  Block  40  pitch  scale.  The  partially  articulated  pitch  scale  was 
preferred  in  high  workload  conditions,  but  the  fully  articulated  pitch  scale  was  better  under  normal 
flight  conditions.  In  Phase  II,  no  differences  were  found  in  pilot  performance  between  the  two  horizon 
lines.  However,  11  of  12  pilots  preferred  the  extended  horizon  line. 

These  results  should  prove  useful  to  System  Program  Office  engineers  as  they  make  decisions 
concerning  the  HUD.  The  results  were  exhaustive;  performance  data,  subjective  workload  data  and 
pilot  opinion  data  were  collected.  Each  source  of  data  lead  to  the  same  conclusion,  thus  providing 
strong  evidence  to  support  the  recommendations. 

Follow  on  efforts  should  look  further  at  alternative  locations  for  the  pitch  numbers.  Several 
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pilots  suggested  placmg  the  pitch  numbers  below  the  pitch  bars  in  the  upper  hemisphere  and  above  the 
pitch  bars  m  the  lower  hemisphere.  This  option  seems  plausible  because  it  gives  the  pilot  an 
instantaneous  indication  of  where  the  horizon  is  located.  Future  studies  should  investigate  this 
possibility  both  with  and  without  articulated  pitch  bars. 

At  least  two  questions  remain  that  cannot  be  answered  by  the  current  effort.  One  area 
concerns  the  best  location  for  pitch  numbers.  The  follow  on  study  mentioned  above  may  answer  this 
question,  Another  question  concerns  the  horizon  line;  if  pilots  prefer  the  longer  horizon  line  because 
it  stands  out  more  on  the  HUD,  then  why  isn’t  their  performance  enhanced?  These  questions  could  be 
answered  via  more  simulation. 


CONCLUSION 

The  data  collected  in  this  evaluation  suggest  that  articulated  pitch  bars  yield  better 
performance  than  the  pitch  bars  presently  used  in  the  F-16.  Specifically,  the  partially  articulated  pitch 
scale  yielded  quicker  reaction  times  and  in  high  workload  scenarios,  this  pitch  scale  induced  less  work 
on  the  pilot.  Of  the  12  pilots,  six  preferred  the  partially  articulated  HUD,  while  six  preferred  the  fully 
articulated  configuration,  Making  a  clear  distinction  between  number  configurations  was  not  possible. 
The  results  were  not  significant  jjind  no  trends  in  the  data  were  apparent.  Similarly,  no  discernible 
differences  were  found  between  the  two  horizon  lines. 

Based  upon  these  conclusions,  the  following  are  recommended.  First,  replace  the  current  F-16 
Block  40  pitch  ladder  with  a  partially  articulated  format.  Pilots  reacted  to  the  partially  articulated 
format  quicker  and  they  know  whether  the  aircraft  is  in  a  nose  high  or  nose  low  attitude  instantly. 
Second,  maintain  the  current  pitch  number  location.  These  results  were  not  compelling  enough  to 
warrant  a  change.  Third,  extend  the  present  F-16  Block  40  horizon  line  the  full  width  of  the  HUD  but 
do  not  occlude  the  altimeter,  airspeed  scale  and  data  blocks.  Pilots  prefer  the  longer  horizon  line. 


* '^Implementation  of  the  third  recommendation  should  be  dune  only  if  recommendation  one  is  imple¬ 
mented. 
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APPENDIX  A 


SITUATION  AWARENESS  RESULTS 

In  the  preflight  briefing,  the  pilot  was  instructed  to  pull  the  trigger  on  the  stic!'.  once  he  felt  he 
had  recovered  to  wings  level.  During  simulation,  the  number  of  times  that  a  pilot  recovered  to  what  he 
thought  was  wings  level,  but  was  actually  inverted,  was  recorded.  Out  of  over  1,700  trials,  the  pilots 
recovered  inverted  only  I'l  times.  The  results,  however,  are  mteresting.  Fifteen  of  the  23  inverted 
recoveries  occurred  using  the  fully  articulated  pitch  scale.  Six  occurred  with  the  Block  40  pitch  scale, 
and  the  remainbg  two  occurred  using  the  partially  articulated  format. 

During  the  S  A  data  analysis,  a  contradiction  was  found  between  the  S  A  results  and  the 
inverted  recovery  results  described  above.  The  following  is  a  description  of  the  problem.  The  bverted 
recovery  results  contradict  the  previously  stated  SA  conclusion  that  the  fully  articulated  pitch  scale  was 
the  best  configuration  (refer  to  Figure  18  on  page  18).  Figure  29  shows  the  percentage  of  inverted 
recoveries  for  each  of  the  three  pitch  scale  formats.  Contrary  to  the  $A  results,  the  fully  articulated 
pitch  scale  yielded  the  most  (65.2  percent)  inverted  recoveries. 

The  results  of  this  evaluation  are  cleur.  Based  on  the  SA  results,  the  pilots  felt  that  their 
situation  awareness  was  very  good  when  flying  with  the  fully  articulated  pitch  scale  while  their 
performance  unequivocally  Indicated  the  opposite.  The  inverted  recoveries  can  be  attributed  to  the 
pitch  ladder  symmetry  found  in  the  fully  articulated  pitch  scale.  Note,  the  partially  articulated  pitch 
scale  (asymmetric  pitch  ladders)  had  the  lowest  number  of  inverted  recoveries.  These  results  indicate 
that,  in  its  present  form,  the  SA  tool  is  not  a  sufficient  indicator  of  pilot  situation  awareness, 
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FIGURE  20,  PERCENTAGE  OF  INVERTED  RECOVERIES  AS  A  FUNCTION  OF  PITCH  LADDER  FORMAT, 
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APPENDIX  B 


PILOT  QUESTIONNAIRE  -  PHASE  1 

1.  Which  pitch  ladder  format  enhanced  your  ability  to  recover  the  most? 

a.  Block  40  pitch  ladder 

b.  pitch  ladder  with  articulated  pitch  bars  m  the  lower  hemisphere 

c.  pitch  ladder  with  articulated  pitch  bars  in  the  upper  and  lower  hemisphere 

2.  Concerning  the  numbers  on  the  pitch  scale,  what  characteristics  are  most  useful?  Choose  all  that 
apply. 

a.  Positive  numbers  in  the  upper  hemisphere  and  negative  numbers  b  the  lower  hemisphere 

b.  All  positive  numbers  with  dashed  pitch  bars  in  the  lower  hemisphere 

c.  Numbers  at  the  outer  ends  of  the  pitch  bars 

d.  Numbers  at  the  outer  ends  and  above  the  pitch  bars 

3.  When  articulated  pitch  bars  are  presented,  would  you  prefer  to  read  degrees  of  pitch  when  the  flight 
path  marker  is  presented  at  the  top  of  the  bars  as  shown  in  Figure  A  below,  or  at  the  bottom  of  the  bars 
as  shown  in  Figure  B  below?  Pick  the  appropriate  letter. 

a. 


b. 
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4.  Have  you  ever  experieaced  spatial  disorientation  using  the  HUD? 

a.  Yes 

b.  No 

5.  Did  the  articulated  pitch  bars  help  in  determining  nose  up  from  nose  do\vn? 

Articulated  bars  in  the  upper  and  lower  hemisphere 

a.  Yes 

b.  No 

Articulated  bars  in  the  lower  hemisphere  only 

a.  Yes 

b.  No 

6.  Were  the  articulated  pitch  bars  helpful  in  determining  roll  attitude? 

Articulated  bars  in  the  upper  and  lower  hemisphere 

a.  Yes 

b. No 

Articulated  bars  in  the  lower  hemisphere  only 

a.  Yes 

b,  No 

7.  For  the  following  HUD  features,  pick  the  one  that  you  prefer  most. 

a.  Nadir  (ground)  pointing  tails 
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b.  Horizon  pointing  tails 
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APPENDIX  C 


PILOT  COMMENTS  CONCERNING  THE  PITCH  LADDERS  AND  PITCH 

NUMBER  LOCATION 

1.  The  following  comments  are  related  to  pitch  scale  preference. 

A.  Articulated  pitch  bars  below  the  horizon  are  great.  .Articulated  pitch  bars  above  and  below 
the  horizon  were  confusing. 

B.  Extreme  nose  high/low  altitudes  were  easier  to  discern  with  articulated  pitch  bars  below  the 
horizon  line. 

C.  No  comment. 

D.  No  comment. 

E.  The  best  display  was  the  partially  articulated  HUD. 

F.  Only  pitch  bars  below  the  horizon  should  be  articulated. 

G.  Fully  articulated  pitch  bars  were  the  best. 

'  H.  Fully  articulated  configuration  was  easiest  to  quickly  determine  which  direction  to  the 
horizon. 

I.  No  comment. 

J.  Fully  articulated  HUD  gave  instant  feedback  on  which  way  to  pull  to  the  horizon. 

K.  No  comment. 

L.  Definite  difference  using  the  full  articulation. 

* 

2.  The  following  comments  are  related  to  the  numbers  on  the  pitch  scale. 

A.  Number  location  really  doesn’t  make  too  much  of  a  difference. 

B.  With  a  partially  articulated  pitch  scale  number  location  does  not  matter,  but  with  a  fully 
articulated  scale,  positive  numbers  in  the  upper  hemisphere  and  negative  numbers  in  the  lower 
hemisphere  is  the  best  choice. 

C.  No  comment. 

D.  No  comment. 

E.  The  best  choice  is  to  locate  the  pitch  numbers  below  the  pitch  bars  in  the  upper  hemisphere 
and  locate  pitch  numbers  above  the  pitch  bars  in  the  lower  hemisphere. 

F.  Would  like  to  see  negative  numbers  in  the  lower  hemisphere. 

G.  No  comment. 

H.  Numbers  at  the  end  tend  to  obscure  the  tic  mark  at  the  end  of  the  pitch  bar  especially  in 
unusual  attitudes. 

I.  No  comment. 

J.  Easier  to  read  when  numbers  arc  above  the  bar.  Also,  this  reinforces  which  way  is  up  and 
v/hich  way  is  down. 

K.  No  comment. 

L.  The  number  placement  currently  used  is  fine. 
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3.  These  comments  concern  pilot  opinions  of  the  flight  path  marker  location  with  an  articulated  pitch 

scale.  ^ 

A.  Easter  to  set  and  hold  desired  pitch  attitudes  by  placing  FPM  between  two  lines  rather  than 
between  two  numbers. 

B.  Having  the  degrees  of  pitch  read  between  the  pitch  bars  distorts  pitch  change  rate. 

C.  No  comment. 

D.  No  comment. 

E.  Having  the  degrees  of  pitch  read  between  the  pitch  numbers  would  be  confusing. 

F.  No  comment. 

G.  No  comment. 

H.  When  dive  angle  is  critical,  the  pilot's  crosscheck  does  not  allow  enough  time  to  estimate  the 
dive  angle  by  looking  at  numbers.  The  pilot  will  note  how  much  he  is  above  or  below  the  pitch 
line  and  correct  from  there. 

I.  No  comment. 

J.  Easy  to  think  about  pulling  to  put  the  FPM  in  the  funnel. 

K.  More  logical  to  "fly"  FPM  through  the  funnel  and  have  the  "wings"  of  the  FPM  point  to  pitch 
angle. 

L.  No  comment. 

4.  The  following  comments  concern  the  utility  of  articulated  pitch  bars  in  determining  nose  up  from 
nose  down. 

A.  No  comment. 

B.  Articulated  pitch  bars  below  the  horizon  prowdes  on  immediate  distinction  between  nose 
high/low  attitudes. 

C.  No  comment. 

D.  Both  arc  good  in  nose  low. 

E.  Difficult  to  tell  what  is  up  and  what  is  down  with  fully  articulated  HUD,  Partially  articulated 
made  it  easy. 

F.  With  partially  articulated  you  get  a  positive  first  glance  indication  of  horizon  position. 

G.  Articulated  below  the  horizon  works  the  best. 

H.  Both  are  better  than  Block  40  HUD. 

I.  No  comment, 

J.  No  comment. 

K.  I  would  like  to  see  partially  articulated  with  numbers  reinforcing  pointing  rails  by  being  on 
the  horizon  side. 

L.  Articulation  was  big  help. 
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5.  The  following  comments  concern  the  utility  of  articulated  pitch  bars  In  determining  roll  attitude. 

A.  No  comment. 

B.  No  comment. 

C.  No  comment, 

D.  No  comment. 

E. They  would  help  in  assessing  the  initial  aircraft  attitude  and  determining  which  way  to  roll. 

F.  No  comment. 

G.  No  comment. 

H.  No  difference  among  all  three  displays. 

I.  No  comment. 

J.  No  comment. 

K.  No  comment. 

L.  No  comment. 

6.  The  folloudng  comoi  Vii  concern  pilot  opinions  of  the  tic  marks  (tails)  located  on  the  outer  ends  of 
each  pitch  ladder. 

A.  No  comment. 

B.  Horizons  are  more  important  than  the  direction  of  the  ground. 

C.  No  comment. 

D.  Very  useful  when  articulation  was  not  available. 

E.  Easier  to  see  horizon  pointing  tails. 

F.  No  comment. 

G.  No  comment. 

H.  No  comment. 

I.  No  comment. 

J.  Easier  to  think  that  the  tails  are  always  pointing  to  the  horizon. 

K.  Have  pitch  numbers  on  horizon  side  of  pitch  ladder  to  reinforce  pointing  tails. 

L.  No  comment. 
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APPENDIX  D 


PILOT  QUESTIONNAIRE  -  PHASE  II 

1.  What  do  you  think  of  the  longer  horizon  line? 

a.  Good  idea 

b.  No  opinion 

c.  Prefer  the  normal  Block  40  horizon  line 

d.  Very  distracting  •  take  it  out 

2.  For  the  following  HUD  features,  pick  the  one  that  you  prefer  most. 

a.  A  ground  pointer  on  the  bank  scale 
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b.  A  sky  pointer  on  the  bank  scale 
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3,  For  the  following  HUD  features,  pick  the  one  that  you  prefer  most, 
a.  45  degree  bank  indicators  around  the  flight  path  marker 
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b.  The  bank  indicator  presently  used 
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4.  Oa  the  bank  scale  shown  below,  the  pointer  rotates  through  360  degrees;  what  is  your  opinion  of  this 
bank  scale? 

a.  Good  idea 

b.  No  opinion 

c.  Prefer  another  bank  scale,  (which  one _ ) 

d.  Would  be  very  distracting 
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5.  Did  you  find  the  bank  scalc/pointer  useful  in  UA  recoveries? 

a.  Yes 

b,  No 

6.  How  do  you  feel  about  (he  pointer  on  the  bank  scale  pointing  up? 

a.  Ver)'  helpful 

b.  Of  some  use  at  low  bank  angles 

«  c.  No  opinion 

d.  Of  little  use 

e.  Bad  idea 

7.  Should  the  bank  pointer  be  limited  to  +  /-45  degrees  of  bank? 

a.  Ycfi 

b.  No 
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APPENDIX  E 


PILOT  COMMENTS  CONCERNING  THE  HORIZON  LINE  AND  THE  SANK 

INDICATOR 

1.  The  followng  cominents  include  pilot  opinions  of  the  long  horizon  line. 

A.  Good  idea  but  not  helpful. 

B.  Should  revert  back  to  a  normal  horizon  line  in  any  weapons  mode. 

C.  No  comment. 

D.  No  comment. 

E.  Gives  a  feeling  of  rolling  faster  than  you  really  are  so  current  horizon  line  is  better, 

F.  Better  reference  to  level  flight 

O.  Good  idea  but  normal  horizon  is  good  enough. 

H.  Easier  to  pick  up  with  peripheral  vision. 

I.  Not  Important  in  day-to-day  operations.  Would  prove  to  be  more  useful  at  night  or  in 
weather. 

J.  Very  nice. 

K.  No  comment. 

L.  Makes  it  easier  to  distinguish. 

2.  Pilot  comments  related  to  the  pointer  on  the  bank  scale  are  summarized  below. 

A.  No  comment. 

B.  A  sky  pointer  always  points  to  the  horizon  and  is  a  positive  indication  of  upright/inverted 
flight. 

C.  No  comment. 

D.  No  comment. 

E.  No  comment. 

F.  No  comment. 

G.  No  comment. 

H.  No  comment. 

I.  Now  make  the  Bank  indicator  360  degrees. 

J.  No  comment. 

K.  No  comment. 

L.  No  comment. 
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3.  The  comments  below  concern  pilot  opinioas  of  two  different  bank  indicators. 

A.  No  iwmment. 

B.  Bank  indication  is  a  minor  concern. 

C.  No  comment. 

D.  The  Block  40  bank  indicator  is  very  useful  in  gear  down  ILS  approaches. 

E.  The  Block  40  Bank  indicator  is  good. 

F.  No  comment. 

O.  No  comment. 

H.  No  comment. 

I.  Bank  indication  around  FPM  oniy  necessary  with  Block  40  HUD. 

J.  No  Comment. 

K.  Bank  indication  around  FPM  would  be  nice  for  flying  in  the  weather. 

L.  Bank  indication  around  the  FPM  would  be  bettor  if  the  indications  were  bigger, 

4.  These  comments  concern  pilot  opinions  of  the  360  degree  bank  scale. 

A.  Too  distracting. 

B.  Bank  indication  is  not  so  important  that  it  requires  this  much  space  on  the  HUD. 

C.  No  comment. 

D.  Make  it  optional  or  incorporate  it  into  the  gear  down  symbology. 

E.  Prefer  current  scale. 

F.  No  comment. 

G.  Prefer  bank  indication  around  FPM, 

H.  Prefer  pointer  below  the  scale  pointing  up. 

I.  No  comment. 

J.  No  comment. 

K.  No  comment. 

L.  No  comment. 

5.  The  following  arc  pilot  comments  related  to  limiting  the  bank  pointer  to  +/-  45  degrees  of  bank. 

A.  Limit  bank,  otherwise  it  would  unnecessarily  cltiltcr  HUD. 

B.  Do  not  limit  bunk,  360  degrees  of  bunk  or  nothing. 

C.  No  comment. 

D.  Displays  go  unnoticed  or  lost  in  bank  angles  greatoi  than  45  degrce.s. 

E.  Limit  Bank,  a  360  degree  bank  indicator  would  be  more  confusing 

F.  No  comment. 

G.  No  comment. 

H.  Useless  for  normal  flying. 

I.  No  comment. 

J.  A  360  degree  display  would  be  nice. 

K.  We  scidom  use  4.‘i  degrees.  A  scale  ranging  bom  +  or  -  30  degrees;  would  be  better. 

L.  No  comment. 
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